THE OPTIMAL HARD THRESHOLD FOR SINGULAR VALUES IS 4/√3 By

We consider recovery of low-rank matrices from noisy data by hard thresholding of singular values, in which empirical singular values below a prescribed threshold λ are set to 0. We study the asymptotic MSE (AMSE) in a framework where the matrix size is large compared to the rank of the matrix to be recovered, and the signal-to-noise ratio of the low-rank piece stays constant. The AMSE-optimal choice of hard threshold, in the case of n-by-n matrix in noise level σ, is simply (4/ √ 3) √ nσ ≈ 2.309 √ nσ when σ is known, or simply 2.858 · ymed when σ is unknown, where ymed is the median empirical singular value. In plain terms, when a data singular value falls below this threshold, this means that its associated singular vectors are too noisy to be used in the reconstruction. For nonsquare m by n matrices with m 6= n the thresholding coefficients 4/ √ 3 (σ known) and 2.858 (σ unknown) are replaced with different constants that depend only on m/n, which we provide. In our asymptotic framework, this thresholding rule adapts to unknown rank and, if needed, to unknown noise level, in an optimal manner: it is always better than hard thresholding at any other value, no matter what the matrix is that we are trying to recover, and is always better than ideal Truncated SVD (TSVD), which truncates at the true rank of the low-rank matrix we are trying to recover. Hard thresholding at the recommended value to recover an n-by-n matrix of rank r guarantees an AMSE at most 3nrσ2. In comparison, the guarantee provided by TSVD is 5nrσ2, the guarantee provided by optimally tuned singular value soft thresholding is 6nrσ2, and the best guarantee achievable by any shrinkage of the data singular values is 2nrσ2. Our recommended hard threshold value also offers, among hard thresholds, the best possible AMSE guarantees for recovering matrices with bounded nuclear norm. Empirical evidence shows that these AMSE properties of the 4/ √ 3 thresholding rule remain valid even for relatively small n, and that performance improvement over TSVD and other popular shrinkage rules is often substantial, turning it into the practical hard threshold of choice. ∗Department of Statistics, Stanford University 1